SOLID robot perception

In a series of papers, we have formalized an active Bayesian perception approach for robotics based on recent progress in understanding animal perception. A central aspect of this active perception approach is the inference of ‘where’ and ‘what’ objects are in the environment under uncertainty, termed Simultaneous Object Localization and IDentification (SOLID). Here we describe some of the details of this algorithm and approach, its extension to include reinforcement learning of the active control strategy and speed-accuracy balance, and how the methods connect with computational neuroscience. In a series of papers [1], [2], [3], [4], [5], [6], [7], [8], we have formalized a Bayesian perception approach for robotics based on recent progress in understanding animal perception. Our formalism extends naturally to active perception, by moving the sensor with a control strategy based on evidence received during decision making. Benefits of active Bayesian perception include: (i) robust perception in unstructured environments [5]; (ii) an order-of-magnitude improvement in acuity over passive methods [8]; (iii) a general framework for simultaneous object localization and identification (SOLID), or ‘where’ and ‘what’ [8]; and (iv) a formalism that naturally integrates with reinforcement learning so that both the active control strategy and the appropriate belief threshold can be tuned appropriately to the contextual situation [6]. Here we describe some of the details of this algorithm and approach, and how it connects with computational neuroscience. The present approach of active Bayesian perception with reinforcement learning for Simultaneous Object Localization and IDentification is further motivated by recent progress in the neuroscience of human and animal perception over imperfect sensor information. Leading computational accounts involve the sequential accumulation of evidence to threshold, consistent with numerous psychological and electrophysiological experiments [9], [10]. Work in computational neuroscience also indicates these principles may relate to the macro-architecture of the brain, principally the basal ganglia and cortex [11], [12]. In particular, it has recently been suggested that the basal ganglia architecture appears configured for optimal decision making over multiple channels of sensory evidence under a wide range of assumptions about what the evidence represents. The correspondence between the basal ganglia anatomy and a network version of sequential analysis holds with generic probability distributions and evidence encoding log likelihoods, log likelihood ratios or the log odds, and is shown in Fig. 1. (The mapping may not be unique: others have been proposed in [11] and [13].) This work was supported by EU Framework project EFAA (ICT-270490). N. Lepora, U. Martinez-Hernandez and T. Prescott are with the Sheffield Center for Robotics, University of Sheffield, UK. Email: {n.lepora, uriel.martinez, t.j.prescott}@sheffield.ac.uk Fig. 1. Map between cortico-basal ganglia anatomy and sequential analysis. (a) A subset of the connections between the cortex and the basal ganglia nuclei. STN denotes the subthalamic nucleus, SNr the substantia nigra pars reticulata and GPe/GPi the globus pallidus external/internal segments. (b) The network version of Bayesian sequential analysis considered in [12], configured to map onto the cortico-basal ganglia anatomy. The striatopallidal indirect pathway (dashed lines) is interpreted as conveying negative changes to decision thresholds−∆Θk , which complement positive changes ∆Θ+k along the direct striato-nigral pathway. Decisions are selected with upwards threshold-crossing in cortex, with the baseline and thresholds shifted consistently to ensure positive neuronal activity. Sequential analysis methods for optimal decision making have also been applied recently to robot perception, focussing on robot touch [1]. A strength of the formalism is that it connects closely with leading work in neuroscience, allowing insights from animal perception to be transferred to robot perception. For example, these methods have enabled the first demonstration of hyperacuity in robot touch [4], giving perceptual acuity finer than the sensor resolution, as is common in animal perception. As discussed above, they also give robust perception in unstructured environments [5] in which there is uncertainty in both where and what objects are, as is also a central aspect of animal perception. Our algorithm for active Bayesian perception with reinforcement learning is based on including a sensorimotor feedback loop in an optimal decision making method for passive perception derived from Bayesian sequential analysis [1]. Sequential analysis uses a free parameter, the decision threshold, to adjust the speed-accuracy tradeoff of the decisions [14]. Our control strategy for active perception also has another free parameter, the fixation point. We thus introduce reinforcement learning to set these two free parameters according to a reward function of the speed and accuracy of the decision outcome. Taken together, this Fig. 2. Algorithm for active Bayesian perception with reinforcement learning. Active Bayesian perception (left) has a recursive Bayesian update to give the marginal ‘where’ and ‘what’ posteriors, allowing active control of the sensor position and decision termination at sufficient ‘what’ belief. Reinforcement learning (right) modifies the decision threshold and active control strategy based on reward information derived from the decisions. results in the algorithm shown in Fig. 2. A key step in our combination of active perception and reinforcement learning is to interpret each active perception strategy (parameterized by the threshold and fixation point) as an action. In other work, we have viewed active Bayesian perception as passive Bayesian perception with a sensorimotor control loop [5], [8]. For passive Bayesian perception, the methods proposed here would optimize just the belief threshold. Thus, by including a sensorimotor loop, the active control strategy is then optimized with the same reinforcement mechanism. In the future, we aim to draw further parallels between the cortico-basal ganglia network for perceptual decision making and algorithms for optimal robot perception, to use insights from animal perception to propose new methods for robot perception and interaction with their environments, while constraining the models of animal perception to be appropriate for robot applications.